2/2/92 test method A.Amin javaheri
Test method I present this framework of test method facets not as a definitive statement or exhaustive list, but rat as a guide for empirical research that I hope will lead to a better understanding of the extent to which these facets affect performance on language tests, and to the discovery of additional facets . The five major categories of test method facet are: (1) the testing environment; (2) the test rubric; (3) the nature of the input the test taker receives; (4) the nature of the expected response to that input, and (5) '''the relationship between input and response. ''Testing environment'' '' '' ''Test environment includes the facets: (1) familiarity of the place and equipment used in administering the test; (2) the personnel involved in the test; (3) the time of testing, and (4)physical conditions. ' ' Test rubric ''' The ''rubric ''of the test consists of the facets that specify how test takers are expected to proceed in taking the test. These include the test organization, time allocation, and instructions. 1. Test organization: The majority of language tests consist of a collection of parts, which may be either individual items or questions, or sub-tests which may themselves consist of individual items. The salience of these parts,' how they are sequenced, and their relative importance can- be expected to affect test takers’ performance. The salience of parts means The test taker’s perception of the test, and hence his performance, may be affected by both the salience of the parts as distinct entities and by the descriptions of these parts that are provided by the test developer. In tests in which the parts consist of individual items, for example, test takers may adopt differing response strategies, depending on a number of factors, such as their perception of the relative difficulty of the different items, how they are instructed to respond, and the amount of time allocated. Sequence of partes means The sequence in which the different parts are presented may also influence test takers’ performance. In '''tests '''which are designed to measure level of ability (‘power tests’), the parts are typically of differing degrees of difficulty, and ordered from easy to difficult. In tests aimed primarily at measuring an individual’s rate of performance (‘speeded tests’), on the other hand, items will be of nearly the same level of difficulty, and may be ordered more or less at random . In tests that consist of separate sub-tests, test takers may not be given the opportunity of answering them in the order in which they choose.In compture test the order in which item are presented is a function of the test taker’s responses. Thus, if the test taker misses a given item, the next item to be presented will be slightly easier, while correct responses will lead to the presentation of progressively more difficult items. Relative importance of parts means the parts of a test are not always weighted equally, with reference to the test taker’s total score. If the test score is the sum' of the individual part scores, the relative importance, or weight, of any given part to the total score will be a function of how much test takers’ scores on that part vary and the degree to which their scores on that part are correlated with their scores on the other parts that make up the test. 2. time allocation: The amount ''of ''time allocated for the test or its parts is likely to affect test performance. In some tests, the time limit is such that not all test takers can manage to answer all the item or ''' '''parts of the test . In speeded tests test takers have limited time to answer items. 3. Instructions: Test ''instructions play a crucial role in test takers’ Performance, since their performance depends, to a great extent, on how well they understand the conditions under which the test will be taken, the procedures to be followed and the nature of the tasks they are to complete. Facets of instructions include: (1) language; (2) channel; (3) the specification of procedures and tasks, and '(4) '''the explicitness of the criteria for correctness. '''Input and expected response The characteristics of the input and the expected response are two additional sets of facets that affect performance on language tests. Input consists of the information contain given test task, to which the test taker is expected to respond . the response is slightly more complex, in that a distinction need made between the expected response and the test taker’s actual response. Its include input format , format of expected response and nature of language. The input format includes the channel,' mod'''e, ''form, vehicl'e, ''and ''language ''of presentation, the ''identification of the problem, ''and the ''degree of speeded ness. Format of expected response include type of response, form of response and language.'' Nature of language input and expected response : When the form of the input or response is language, that language can be characterized by its length, ''propositional content, organizational characteristics, ''and ''illocutionary Characteristics. Length: Input and responses that consist of language samples may vary in length from a single word, to''' a sentence, to an extended piece of discourse '''propositional content; Propositional content can be described with reference to the characteristics of the information in the context and in the discourse :vocabulary, degree of contextualization, distribution of information, type of information, topic, ''and ''genre. Vocabulary: The vocabulary used in the input and response can vary in a number of ways, we might expect that the less frequent the vocabulary used in the input, the more difficult the task will be. Degree of contextualization : Cumins (1983) '''has described the notion of ‘context embeddedness’'' as one characteristic that differentiates typical non-academic interactive language use from academic language use. ''Distribution of new information: ''This facet characterizes the distribution of the new information that must be processed and manipulated in order for the test taker to successfully complete a given test task. Discourse in which new information is distributed over a relatively short space or time may be called ''compact, ''while discourse with new information distributed over '''a relatively long space or time may be called ''diffus'e. Type of information: The type of information in input and expected response can be classified along three dimensions: concrete abstract, positive negative, and factual counterfactual. We generally think of abstract' ''information as that whose mode of representation is primarily symbolic or linguistic, whereas ''concrete ''information is capable of representation in other than linguistic modes, such as visual, auditory, tactile or kinesthetic. Information can also vary in terms of the degree to which it is negative. Information that includes no explicit negative elements or markers can be considered to be positive. Counterfactual information is that which includes assertions about conditions which are possible or probable in some alternative world, but not in the known, factual world. '''Topic: The topic of a given piece of discourse is generally understood as‘what it is about. Genre: Hymes (1972a) uses the term ‘genre’ to refer to a component of speech that has identifiable ‘formal characteristics that are traditionally recognized’, giving as examples the categories of ‘poem, myth, tale, proverb, riddle, curse, prayer, oration, lecture, commercial, form letter, and editorial. ' ' Organizational characteristics Organizational competence as comprising those abilities that are related to controlling the formal organization of language. This formal organization is a characteristic of the course of both the input and the response, and three is three types: grammar, cohesion, and rhetorical organization. ' ' Pragmatic characteristics '' '' '''''Illocutionary force The input and response can also be characterized in terms of their illocutionary force, or the language functions performed. As noted by Searle (1969), the function performed by any ‘exam question’ is not to elicit information, but rather to find gut if the examinee knows something.. Input consisting of a language sample thus performs the ‘primary’ function of requesting a response and could, in this somewhat trivial sense, be regarded as functional. Sociolinguistic characteristics: The last set of facets by which the language of the input an4 response are characterized are sociolinguistic characteristics, which consist of (1) dialect or variety; (2) '''register; and (3) naturalness. '''Relationship between input and response The relationships between input and response in language tests can be classified into three types: reciprocal, nonreciprocal, ''and ''adaptive. Reciprocal input and response: Reciprocal language use can be defined as the use of language by one individual to produce an effect in another individual through the reduction of uncertainty with knowledge of results. Nonreciprocal input and response:'''''Nonreciprocal language use is that in which there is no interaction between language users, that is, in which continual give and take between two or more individuals does ''not ''take place. In nonreciprocal language use, therefore, there is no feedback, and the language used does not affect subsequent language use. In reading a book, for example, the reader is seldom able to give feedback to the author, and the text of the book will not change, irrespective of whether or not the reader finds certain statements objectionable or of questionable accuracy and notes this in the margins, or needs to stop and look up a word in a dictionary. '' '' ''Adaptive input and response: ''The relationship between input and response is adaptive if the input '' ''is influenced by the response, but without the feedback that characterizes a '' reciprocal relationship. In an adaptive test the particular '' '' tasks presented to the test taker are determined by her responses '''to previous tasks.